System and method for synthesizing spoken dialogue using deep learning

ABSTRACT

Disclosed are a system, method and software to associate attributes with digital media assets using deep learning. Digital media contains specific assets, such as audio, that can be replaced with other assets. The system, method and software allow for personalizing digital media content based in part on neural network analysis in order to generate synthesized audio, such as speech, spoken prompts, and/or dialogue.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of and claims priority to U.S. patentapplication Ser. No. 16/840,337, entitled “Method And System ForDelivering Personalized Content As Augmented Reality Overlays,” filedApr. 4, 2020, U.S. patent application Ser. No. 15/333,326, entitled“Method, System and Software for Associating Attributes within DigitalMedia Presentations,” filed Oct. 25, 2016, and U.S. patent applicationSer. No. 14/506,822, entitled “Method, System and Software forAssociating Attributes within Digital Media Presentations,” filed Oct.6, 2014, each of which claim priority to U.S. patent application Ser.No. 10/819,514, entitled “Method, System and Software for AssociatingAttributes within Digital Media Presentations,” filed Apr. 7, 2004,which claims priority to U.S. Provisional Application No. 60/460,998entitled “Method and System for Digital Entertainment AssetPersonalization,” filed Apr. 7, 2003, which are incorporated herein byreference in their entireties. This patent application is also relatedto U.S. patent application Ser. No. 10/819,404 entitled “Method, Systemand Software for Digital Media Narrative Personalization,” filed Apr. 7,2004, which issued as U.S. Pat. No. 8,478,645 on Jul. 2, 2013.

BACKGROUND OF THE INVENTION

It has been stated that human thinking falls into one of two categories:reasoning and narrative, descriptive, contemplative thinking. Reasoningis the process that permits the acceptance of facts followed by alogical process to conclude a result, opinion, conclusion or decision.Narrative, descriptive, contemplative thinking corresponds to thestorytelling process (including receiving of a story) and the thoughtprocesses associated with the reception of a story.

Text and stories (presented in various formats) can also be interpretedat various levels including the literal, ethical, historical andmystical levels. It is generally understood that texts and stories areread and interpreted in different ways including at the differentlevels. A story or narrative can have different meanings at differentlevels, different interpretations at different levels, and will beinterpreted differently by different individuals at each of the levels.Additionally, an individual, based on their emotional or physical state,may interpret each level of a story differently. A person may reactdifferently to a story based on their current emotional state, which mayin fact change while they are receiving the story. In some cases, it maybe possible to alter their emotional state while receiving the story tomake them more receptive to a particular interpretation at one or morelevels.

Advertising is typically a type of narrative in which the advertiserattempts to influence the recipient (viewer, reader, listener) in a waythat creates a positive association with a product or in some casesattempts to influence the recipient to purchase the product. Humor isfrequently used in advertising because it puts the recipient in areceptive state for a message regarding the product. As such, humor isalready used to alter the emotional state of the recipient of theadvertising. Such manipulation of the emotional state of the recipientis currently limited to simple manipulation through the presentation ofthe advertisement (narrative).

With the advent of the digital era, the number of ways to transmit andpresent advertisements has increased dramatically, with advertisingbeing inserted into Web pages, being displayed as pop-up windows on Webpages, transmitted as streaming media over the Internet, presented asanimation over the Internet, and inserted into television streams atcentral or distributed insertion points. The advent of the digital eraalso threatens advertising in that viewers can avoid advertisements bystopping pop-up windows, or skipping over the advertisements on aPersonal Video Recorder (PVR).

Due to the rapid changes in technology, advertisers are being forced totake new approaches to advertising including product placement, in whichproducts are advertised during an entertainment narrative (e.g. asit-com) as opposed to in a separate advertisement. Advertisers are alsoexploring ways to draw viewers (or listeners) to content that isessentially digital media narrative storytelling, typically in the formof music or a short story that would appeal to viewers. What is requiredare ways to attract individuals to content that is personally morerelevant and impactful for them and which may contain an advertisingmessage (in the form of product placement), and have them receive thatmessage in full, as opposed to skipping over all or a portion of themessage. It would be an additional benefit to have the ability tounderstand the individual's likes and dislikes or current mood in orderto adapt the message appropriately for the individual at the time thatthey are receiving it.

Digital media narrative may include content, such as video, audio, andcombinations thereof, that is distributed digitally and can be storeddigitally. In some instances digital media narrative can be manipulatedvia computer without human interaction. Digital media narrative includesbut is not limited to stories presented over the Internet, modifiabledigital video including digital television and streaming media,presentations on web sites including animations, still images, sequencesof still images, audio, textual presentations, and combinations of allof the abovementioned media.

Digital media narrative can be distributed in a multitude of ways, suchas broadcasting over cable and satellite, the Internet, or on media suchas optical disks (CDs or DVDs). Distribution of digital media narrativeis a significant business and involves complex relationships betweenartists and record labels or film studios, producers, and a completedistribution chain that includes advertising, both in the form ofadvertising of the content itself as well as sponsored advertising thatmay be inserted into, or become part of the content such as productplacement. Examples of sponsored advertising include the insertion ofcommercials into recorded materials or the use of sponsor's products(e.g., automobiles) in the narrative.

Digital media narratives may be directed at a segment of the market, butthe segment that the digital media narrative material is directed to maybe quite large. For example, an artist may create works that appeal tothe 18-24 year old age group, but that group can include people withdifferent interests from various countries and different economicclasses.

Furthermore, when digital media narratives are used as part ofadvertising, that digital media narrative and the advertising may bedirected at such a large market segment that it is difficult to keep theinterest of individuals within that market segment. If a user goes to anInternet web site to view a recording of an artist, and in particular asponsored work that contains advertising, that work (including theadvertising) may be so generic that the user cannot respond to or relateto the work. Additionally, the user may experience different emotionalstates than another user, or the user may be in an emotional state thatis different than the user's emotional state when the user previouslyviewed the material.

For the foregoing reasons, there is a need for a method, system, andsoftware to enable a distributor of sponsored or un-sponsored digitalmedia narratives to change the content of the digital media narrativebased on user demographics, psychodemographics, emotional states,affinities (cognitive, emotional and social), self-narrating contentclassification, internal narrative traits preference topology, timesensitive, episodic expectation sequencing, and collective/collaborateattributes.

SUMMARY OF THE INVENTION

A method, system and software are presented which allow for customizingand personalizing content based on a combination of the user'sdemographics, psychodemographics, cognitive states, emotional states,social placement and group interaction dynamics within an onlinecommunity, and/or affinity for certain content elements (images, sounds,segments, graphics, video, text, dialog), self provided narratingcontent, internal narrative traits preference topology, and expectationlevel and temporal spacing of assets within the narrative. In oneembodiment, a user selects material for presentation and is presentedwith the dynamic digital media narrative and optional embeddedadvertising presented in a customized format that gives the user apersonalized experience.

The system, method and software may support product placement typeadvertising by modifying both the product placement material as well asthe narrative surrounding the product placement material. Appropriateadvertising may be selected as well as selecting components for thenarrative that supports the product placement advertising.

In one embodiment, the user accesses a website containing sponsoredmaterial and logs on to that website. Information regarding thedemographics, psychodemographics, cognitive states, emotional states,social placement and group interaction dynamics with the onlinecommunity, and/or affinity for certain content elements of the user isretrieved from storage. If the user is a new user, he or she is asked aseries of questions, preferably through a form of media narrative, thatwould creatively ask the questions, in order to obtain key informationthat the system would then classify and include into the user's profile.These questions can include yes/no, multiple choice, like/dislikesratings or any other type of personal assessment or individual and/orgroup question. Alternatively, the user may interact with the medianarrative and the interactions may be used to create a user profile.

In one embodiment, the user requests that the material be played ordisplayed, and is simultaneously presented with that material andcustomized personalized graphics and video. In another embodiment, theuser can interact with the graphics and video. By making certainchoices, a profile of the user can be determined and updated and thecontent (including background material, primary video, overlay graphicsand audio) modified appropriately, thus giving the user an enhancednarrative, and more meaningful and emotionally satisfying experiencewith respect to the viewed content. In both embodiments, the userprofile can then be used to present the most appropriate digital assetsto the subscriber, namely those with which the user has the highestaffinity, or those which map well to the user's tendencies andtemperament, which may be included in the user's internal narrativeperception identification framework. The internal narrative perceptionidentification framework may include a collection of attributes,qualities, and measurements regarding the user that allow for matching,correlation and/or selection of digital media assets that areappropriate for that user and the effective communication of themessage.

In one embodiment, the user's demographics are used to determinepersonalized content. Demographics may include, and are not limited to,group affiliations, regional location, sex, sexual preference, age,family situation (married or single, children or no children), race orethnic background, religion and socio-economic condition. The user'sdemographics may be used alone, or in combination with the previouslydiscussed techniques, to create the individual internal narrativeperception identification framework.

One advantage of an embodiment of the method, system and softwarepresented herein is that the user is presented with an enhancedexperience of the creator's content that creates a greater emotionalexperience for the user and a more impactful narrative. This creates amore receptive state for product placed advertising inserted into thecontent, for a direct advertisement, or for entertainment containing anadvertising message. Another advantage of the method, system andsoftware presented herein is that it allows advertising to be insertedin subtle ways and presented in a context in which users may be able tofully engulf themselves into the lifestyle being positioned andportrayed by the brand. By combining the personalized digital medianarrative asset elements with product placement advertising, a morepowerful media is created for the advertisers and the artists.Additionally, since users typically volunteer to see the content,including advertising, they are much more likely to be receptive to themessage presented, and less likely to skip over or fast-forward throughthe content including the advertising.

In an alternate embodiment, users are presented with questions or chatlike text based opportunities within a narrative to either share theemotional experience with other users or to comment on their ownemotional experience. This enables a user to share an experience withfriends or acquaintances or allows a user to “sit back and relax” whilesomeone else controls the enhanced experience. In this embodiment, thepersonalized content is based on the learned social dynamics of theuser.

Another embodiment includes a developer toolkit that allows a creativedirector to create trigger points in the material where digital assetssuch as overlay graphics, backgrounds, icons, text, sound, and/orproduct placement images can be inserted into the digital medianarrative. The developer toolkit allows the creative director to createa map of the elements of the user's profile and his or her internalnarrative perception identification framework topology to digital assetsthat may be created. The developer toolkit also provides for the abilityto insert interactive material such as questions or interactivegraphics, allowing for the updating and enhancements of the user profilebased on the user's interactions. The toolkit further permits thecreative director to create and link the digital assets to the materialsuch that assets for which the user has a high degree of affinity orassets having a highest probability for matching a user's internalnarrative perception identification framework topology can be selectedat the trigger point.

One advantage of an embodiment of the method, system and softwarepresented herein is that they may enable a central server or service tocompile a user's internal narrative perception identification frameworkand to store the internal narrative perception identification frameworkas an internal narrative perception identification framework profile andallow this profile to track or follow the user as he or she perusesmultiple digital media narratives. This enables the user to receiveenhanced digital media assets and various narrative experiences andallows the service to be sold to advertisers that advertise on any website or broadband IP-based distribution point.

These and other features and objects of the invention will be more fullyunderstood from the following detailed description of the embodimentswhich should be read in light of the accompanying drawings.

In this respect, before explaining at least one embodiment of theinvention in detail, it is to be understood that the invention is notlimited in its application to the details of construction and to thearrangements of the components set forth in the description orillustrated in the drawings. The invention is capable of otherembodiments and of being practiced and carried out in various ways.Also, it is to be understood that the phraseology and terminologyemployed herein, as well as the abstract, are for the purpose ofdescription and should not be regarded as limiting.

As such, those skilled in the art will appreciate that the concepts uponwhich this disclosure is based may readily be used as a basis fordesigning other structures, methods, and systems for carrying out theseveral purposes of the present invention. It is important, therefore,that the claims be regarded as including such equivalent constructionsinsofar as they do not depart from the spirit and scope of the presentinvention.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and form a part ofthe specification, illustrate embodiments of the present invention and,together with the description serve to explain the principles of theinvention.

FIG. 1 illustrates a user-relationship diagram for an embodiment of thesystem;

FIG. 2 illustrates the representative components of a digital medianarrative and the digital media assets;

FIG. 3 illustrates the creation of a personalized digital medianarrative;

FIG. 4 illustrates the use of interactivity to create an enhanced userprofile and an enhanced experience through a personalized digital mediaasset;

FIG. 5A illustrates a context diagram for digital media narrative assetpersonalization;

FIG. 5B illustrates potential databases that may be used in creation ofthe personalized digital media asset and the relationships between thosedatabases;

FIG. 6 illustrates exemplary relationships between the social,emotional, and cognitive affinity elements;

FIG. 7 illustrates exemplary components of the cognitive element;

FIG. 8 illustrates exemplary components of the social element;

FIG. 9 illustrates exemplary components of affinity;

FIG. 10 illustrates a flowchart for the development of the componentsfor a personalized digital media narrative;

FIG. 11 illustrates data structures for a relational database forlinking components of a personalized digital media asset withaffinities;

FIG. 12 illustrates an object oriented approach for linking componentsof a personalized digital media asset with affinities;

FIG. 13 illustrates a computer on which the invention can be built;

FIG. 14 illustrates the construction of an Interactive Musical Intersode(IMI);

FIG. 15 illustrates a registration screen for an IMI;

FIG. 16 illustrates a screen representing a personalized background inan IMI;

FIG. 17 illustrates a screen with personalized content in digital videoform;

FIG. 18 illustrates a screen posing a question related to an IMI to auser;

FIG. 19 illustrates a screen posing a multiple choice question relatedto an IMI to a user;

FIG. 20 illustrates an alternative screen for posing a question relatedto an IMI to a user;

FIG. 21 illustrates a screen having a text communication window relatedto an

IMI;

FIG. 22A illustrates a flow chart for the operation of the screens shownin FIGS. 15-21;

FIG. 22B illustrates a flow chart for the operation of a fuzzy logicbased Enhanced Director Agent (EDA); and

FIG. 23 illustrates a narrative perception identification framework thatenables matching of digital media assets to individuals.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

In describing an embodiment of the invention illustrated in thedrawings, specific terminology will be used for the sake of clarity.However, the invention is not intended to be limited to the specificterms so selected, and it is to be understood that each specific termincludes all technical equivalents which operate in a similar manner toaccomplish a similar purpose.

With respect to the user, a number of terms are used that describe howthe user is identified and profiled. As used herein the term “useridentification” means a number, symbol, or alphanumeric sequence thatidentifies a single subscriber, a group of subscribers, a subset ofsubscribers, or a subscriber in a specific location or on a specificnetwork.

The term “user profile” includes a stored or calculated profile thatdescribes one or more aspects of the user, such as demographics,psychodemographics, and attributes. The profile can be determined fromquestions answered by the user, forms filled out by the user, and/orinteractions of the user with the digital media narrative.Alternatively, the profile can be determined from the user's web surfingcharacteristics, shopping habits, television viewing habits, and/oractual purchases. Profiling of users based on these interactions,viewing habits, and purchases is well understood by those skilled in theart.

“User attributes” include aspects, characteristics, and qualities of theuser that are useful for determining (matching, correlating, andselecting) digital media assets. These attributes may includecharacteristics such as affinities, likes or dislikes as describedoutside of affinities, perceptions, experiences, and other factors thatplay a role in determining the internal narrative perceptionidentification framework.

The term “internal narrative traits preference topology” means arepresentation of personality, such as a representation that is similarto the Myers-Briggs personality classification scheme. However, therepresentation establishes a measure of the potential for impact on theindividual as specifically applied to narrative and interactivenarrative. The Keirsey temperament sorter is a personality test thatscores results according to the Myers-Briggs personality classificationscheme, and allows testing or classification to occur over the Internet.The Keirsey test allows viewers to create a personality inventory ofthemselves. The internal narrative traits preference topology thusprovides for personality classification and gathers data from a varietyof sources, including the individual's interactions with the interactivenarrative. As can be seen in FIG. 23, the personality classificationdeals with personality traits such as engager v. bystander, identifierv. detacher, metaphorical v. literal. Other classifications can beincluded to further define a personality. Defined topology is theequivalent of the internal narrative traits preference topology.

“Narrative content” includes content used for story telling includingstory telling containing direct advertising, product placementadvertising, or combinations thereof “Story based” content refers tocontent that tells a story, which either is based on fact or isfictitious in nature, as opposed to a simple recitation of productcharacteristics. This is not to imply that advertisements that containproduct characteristics or facts cannot be story based, only that thestory based content contains those characteristics or facts in thecontext of a story.

“Trigger points” are occurrences or time points within a story ornarrative content that may cause the recipient (viewer, reader, orlistener) to take a particular interpretation at one or more levels orthat may affect the user's emotional state. “Personalization triggerpoints” are those trigger points that allow for modification of thestory or narrative content in support of customization of the content tomatch the internal narrative perception framework and appropriatelyinfluence the user.

“Episodic” content refers to narrative content or stories that containepisodes, which either are arranged in a time sequence or are accessibleto the user individually. A time sensitive sequence or set of episodesincludes narrative content or stories that create personalized impact iftemporally or sequentially changed. For example, a time sensitivesequence may include a mystery or story with a surprise ending that, perindividual, may vary with respect to timing of delivery (i.e., wait 3seconds versus 5) to build the optimal level of anticipation perindividual. FIG. 4 illustrates other aspects of time sensitive episodic& expectation sequencing.

“Self-narrating” refers to interjecting elements of a user's life intothe narrative that the user is watching. Self-narrative techniques takestories related to the user's life in a deeper way. Users can provideself-narrative content that may be incorporated as narrative or digitalmedia assets that may be included in personalized narrative. As anexample, a user may upload a photo of himself and that photo may be usedin a personalized narrative being presented to another user. This mayoccur when a user accepts membership in a group and agrees to share aviewing experience with another user. Self-narrative content may includeaudience-generated content. This content may be self-sustaining in thatit can automatically be incorporated into numerous personalizednarratives and add another life or perspective to the narrative, asopposed to simply being inserted or viewed at the direct request of auser.

A “self narrating audience generated content classification” refers tothe labeling of a digital media asset such that the individual or groupgenerating the asset is known, as well as the potential for using thatasset. As an example, a user may provide a photo of himself, andindicate that that photo can be inserted into narrative presented to aselect group of users or potentially all users. Similarly, the user mayupload other digital imagery, text, video, or other material that servesas a background, an overlay or another element in a narrative. Theclassification of this material determines how and when it may be used.

“Significance of affinity” is a measure of the strength of affinity andis useful in determining the level of attraction an individual has for aparticular element. As an example, a user that repeatedly selectsoranges in an interactive presentation would have a higher probabilityof having a high level of affinity for oranges. The degree ofsignificance may be related to the probability of having an affinity foran element.

“Association rules” provide the ability to match digital media assets toan individual, through a correlation of the attributes of the asset withthe assets of the individual in order to provide the highest level ofimpact. The correlation of the attributes may consist of summing thenumber of matching attributes, identifying key attributes, or providinga true/false test for one or more attributes. A relative weightingscheme may be incorporated into the correlation to give preference to oremphasize certain attributes. Correlation thus refers to the process ofmatching or selecting a digital media asset based on the overlap betweenthe attributes of the individual and the asset, with the goal of havinga greater narrative impact.

A “collective/collaborative classification,” as illustrated in FIG. 22B,describes one or more attributes related to an individual's dynamicswithin the community and the potential for content to be defined withinthat dimension. Collective/collaborative attributes are those specificmeasures of the individual's dynamics within the community.

An “affinity” is a measure of how much an individual is attracted to aparticular element of a narrative. As an example, an individual who wasalways served fried eggs on Wednesdays by his/her mother may have fondmemories of that experience and may have a positive affinity for friedeggs, in particular on Wednesday. Conversely, an individual who wasforced to eat cold fried eggs as a punishment may have a negativeaffinity for fried eggs.

FIG. 1 illustrates a detailed diagram for an embodiment of the system,in which a creative director 110 creates the personalized content anddistributor 102 distributes the personalized content that is viewed by aconsumer 104. Although FIG.1 illustrates the creative director 110 andthe distributor 102 as separate individuals or entities, the creativedirector 110 and the distributor 102 may be the same person/entity andhave different economic arrangements with other users of the system. Thecreative director 110 works with the content assembly and distributionsystem 100 to take original content 114 produced by an artist 112 andcreate modified content 106, which is viewed by a consumer 104.

In terms of economic arrangements, in one embodiment, a sponsor 101 mayprovide financing 108 to one or more of the artist 112, the creativedirector 110 and the distributor 102, such that they have resources toprovide media assets and to provide a viewing mechanism for the consumer104. In one embodiment, the artist 112 is a musician who provides songsand musical videos as original content 114. In this embodiment, thesponsor 101 is a manufacturer, such as a manufacturer of consumerpackage goods who desires to advertise its products to the consumer 104.The sponsor 101 provides financing to the artist 112 and the creativedirector 110/distributor 102 to permit the creative director110/distributor 102 to create a database system and the appropriatedigital media assets for personalized viewing by the consumer 104.

In an alternate embodiment, the sponsor 101 finances the artist 112 toprovide new material directly related to its product, which mayincorporate elements of previously produced songs, videos or otherartistic works. In this alternate embodiment, the sponsor 101 may retaina closer relationship with the artist 112 to produce contentspecifically for advertising its products. This content may be furtherpersonalized through the content assembly and distribution system 100 toproduce modified content 106 for the consumer 104. This modified content106 is the personalized digital media asset.

Note that in the description set forth above, the artist 112 need notdirectly interact with or be financed by the sponsor 101. Rather, one ormore intermediaries, such as agents, producers, studios, distributors orother entities may represent the artist 112 to the sponsor 101.Accordingly, the term “artist” is intended to include an actual artistand/or one or more such intermediaries.

FIG. 2 illustrates the representative components of a digital mediaasset including background images 200, video sequences 204, foregroundimages 202, text 208, branding graphics 206 and digital audio 210. Theseelements may be combined to create digital media assets 212.

Background images 200 for digital media assets 212 may be still graphicimages or MPEG based images, desktop colors, flash files, patterns orany other types of images that are typically used as background elementswithin current or future digital media narrative experiences. Videosequences 204 may be in the form of MPEG-2 or MPEG-4 video sequences,but may also be other types of video including but not limited to RealVideo, AVI video, flash based video, animations, or other videosequences. Text 208 may include overlay text scrolling, masked text,and/or other types of textual messages that appear on a screen, such astickertape messages that appear at the bottom of a screen. Brandinggraphics 206 may include icons, symbols, figurines or any other types ofelement that appear on a screen that are not typically thought of asvideo images. Foreground images 202 may include still images, such asphotographs, drawings, animations or any other types of image, that arebrought to the attention of a user in the foreground. Digital audio 210may be any type of digital audio including MP3 audio or any othercompressed or uncompressed digital audio streams, files or segments.

Although personalized digital media assets 212 are shown to haveindividual sets of frames, it is to be understood that the digital mediaassets may be a combination of all of the above aforementioned elementsand may or may not contain individual frames, but may have certainpoints which delineate segments or pieces of the asset from the pieces,both temporally and in terms of content.

FIG. 3 illustrates the creation of a personalized digital media assetsbased experience. In FIG. 3, an initial digital media asset videosequence 300 is combined with music and lyrics 302 to create a modifieddigital media asset that is personalized or internal to a user. Thispersonalized user story 306 may contain images, audio, text and/or othercontent 304 that is expected to elicit certain emotions in the mind ofthe user.

An affinity may be the extent to which something creates an impact onthe user, based on the user's life experiences. As an example, an oldT-shirt may remind a user of an individual that previously wore thatT-shirt. This may be a strong affinity if that person was verysignificant to the user (e.g., a parent, a spouse or a child). In somecases the user may not have an affinity for that object but maysubsequently develop an affinity based on something that the user sees.As an example, if the user sees a magazine cover with an attractiveperson wearing an old T-shirt, this may create a new affinity. Asanother example, hearing a song while on a first date with a futurespouse may create a new affinity to that particular song or artist. Theuser may have strong emotional feelings upon hearing the song becausethe song prompts the user to recall the early dating experience.Affinities and the quality of affinity thus define the extent to whichthe object or, in the case of the present system, digital media assetshave an impact on users. Since affinities constantly change and maydepend on a user's emotional state as well as past experiences, thesystem may update user profiles to determine which digital media assetsshould be used to create stronger emotions in the user.

The personalized (internal) user story 306 may be what the userperceives the digital media asset to be and may depend, at least inpart, on the emotional state, demographics, psycho-demographics,cognitive states, social placement and/or group interaction dynamicswithin the online community, affinity for certain content featuresand/or other factors particular to the user. Trigger points 320 may bepresented such that the digital media assets are customized to providethe user with a new personalized (internal) user story 312. This newpersonalized (internal) user story 312 may be composed of new images,audio, text and/or other content that, based on a user profile, areexpected to trigger affinity element emotions 310 that are newer and/orstronger than emotions 304 previously experienced by the user or theexpected emotions if the media assets were not personalized.

Trigger points 320 provide the mechanism for content management and thecreation of a more personalized digital media asset based on a user'spersonal experiences. Trigger points 320 can be placed at various pointsin the digital media content, based on determinations of the creativedirector 110/distributor 102. For example, the creative director110/distributor 102 may decide to place trigger points 320 in thedigital media content 300 so that they occur at various points in time,when a certain character appears on the screen, when certain text isdisplayed, when words are spoken or sung, or based on other features ofthe digital presentation. During presentation of the digital mediacontent, when the presentation reaches a trigger point 320, a script orother software program is executed. The creative director 110 or thedistributor 102 may customize the script for each trigger point 320. Thescript may cause a computing device to access a database containingprofile data relating to the user, and, based on the user profileinformation, the script may cause the insertion and/or replacement ofvideo, graphics, audio or other material in the digital presentation.Furthermore, the timing of the presentation including but not limited toplayback speed or changing the sequence of the presentation may bealtered.

For example, the creative director 110 may place trigger points 320 in adigital media asset video sequence 300 each time that a particularcharacter appears on screen. When the character appears on the screen,the script may, for example, access the user profile database todetermine whether the user responds favorably to the character. If theuser responds favorably to the character, the script may cause theinsertion of a product advertisement on a location in the video screenwhenever the character appears on the screen. Conversely, if the userdoes not respond favorably to the character, the server may take noaction, or the system may insert different content into the screen whenthe character appears. Another example of a trigger point 320 is aportion of the digital media content that contains “hidden”advertisement, such as a character driving a particular brand ofautomobile while background music is played. The trigger point in thiscase could include both the type of vehicle and the background music.Upon reaching the trigger point 320, a script is run determining whichtype of vehicle manufactured by the sponsor may connect with the userand which song among a group may have the most appeal or greatestaffinity to the user. For example, certain users may have a strongconnection to pickup trucks and country music while other users mayrespond to Sport Utility Vehicles (SUVs) and classic rock. Upon reachinga trigger point 320, the system may select the make of vehicle that thecharacter drives, the type of music played in the background and thevolume of the background music. The system could also access additionalinformation outside of the user profile to determine whether insertionshould occur. For example, if a first user's profile indicates that theuser has an affinity for a second user, when the character appears onthe screen the system may determine whether the second user is alsoviewing the presentation at the same time. If so, the system may placean icon on the screen to indicate to the first user that the second useris also viewing the presentation.

In addition to trigger points 320, flexible trigger points 322 may beutilized. Flexible trigger points 322 have the property that they may bemoved in time or entirely deleted. Flexible trigger points 322 thusallow further personalization of the digital media asset experiencebased on specific assets of the user typically learned through typicalactivity of the user with digital media asset 212. Trigger points 320and flexible trigger points 322 may include time stamps and triggerpoint IDs 324. As represented in FIG. 3, time stamps and trigger pointIDs 324 may provide an indication of when the trigger point occurs 322and the affinities associated with that user.

FIG. 4 illustrates a default experience 400 being shown to a user withinteractive opportunities 402 in conjunction with trigger points 320.The trigger points 320 may be used to present interactive opportunities402 to the user. Based on the user's responses to these interactiveopportunities 402, an enhanced user profile 404 may be created. Theenhanced user profile 404 may subsequently be used in conjunction withtrigger points 320 to create a personalized experience 406. Personalizedexperience 406 may include different content which is better suited tothe user's demographics, psycho-demographics, cognitive states,emotional states, social placement and/or group interaction dynamicswithin the online community, and/or affinity for certain contentelements.

For example, the default experience 400 may be a new music video havinginteractive opportunities 402 that include selecting whether the videocontains scenes from Spain or Italy followed by selecting scenes fromNew York City or the Rocky Mountains. The user's selections may bestored and utilized in personalizing future presentations. For example,if a user selects that a character travels from Italy to the RockyMountains, the system may infer that the user enjoys mountain sceneryand perhaps skiing. Upon viewing an advertisement for a soda, thedigital content may include a skier stopping to drink the soda.Conversely, if the user preferred New York City, the digital content maycontain a dancer at a club stopping to drink the soda.

In another embodiment, a new user is asked a series of yes/no questions,multiple choice and/or open-ended questions. For example, a user may beasked to answer questions within the context of the narrative such as:“What is your favorite animal?”; “Would you rather ride a motorcycle ordrive a sports car?”; “Do you prefer blue, red, green or yellow?”; or“True or false, I like fishing?” Questions may include demographicquestions, such as gender, age, ethnic background, income, educationlevel and region of residence. A user may also be asked generalquestions about his or her mood, state of mind or personality traits.The answers may be compiled to create a user profile which includesdemographic information and personal likes/dislikes. The demographicinformation may be used to select the appropriate general database towhich a member belongs, such as young female or middle-aged male. Thequestions may be designed to gauge the user's personality traits andaffinities, the user's emotional state and the user's emotional responseto media content. The queries may be more abstract than directquestions, such as “Pick a happy color,” “Choose a word that saddensyou,” or even something similar to projective psychological tests suchas an ink blot test or word association test.

Demographic information may be compiled because an individual'sdemographics have a great affect on his or her interest. A middle-agedparent is more likely to be interested in family oriented medianarrative while a single young-adult is more likely to be interested inmore risque media narrative. Also, a user that is in the economicmiddle-class may be more interested in high-priced leisure activitiessuch as golf or skiing, while a user that is in a lower economic classmay be more interested in less costly activities, such as basketball.Compiling an individual's demographics lends a wealth of information onmaterials that are more likely to have a personal connection.

FIG. 5A illustrates a context diagram for one embodiment of a digitalmedial narrative asset personalization system (server) 590. In thisembodiment, a user 501 receives the assets that comprise thepersonalized digital media asset 212 (FIG. 2), and supplies a user ID520, a password and interactions/choices.

The server 590 may develop the personalized digital media asset 212 fromcontent 531 and the digital asset repository 541. In one embodiment, theserver 590 stores modified content in a modified content storage medium551. In an alternate embodiment, the modified content is not stored, andthe personalized digital media asset is presented to user 501 via theserver 590.

Referring again to FIG. 5A, the user may also participate in an onlinecommunity 521 providing a user with the ability to interact with otherusers. In one embodiment, the online community 521 includes the abilityto share a viewing or listening experience with another user, thuscreating new affinities for the content of that viewing or listeningexperience.

In one embodiment, the user 501 provides a user ID 520, a password 522,and interactions/choices 524 to a server 590. The user 501 may bepresented with digital video 500, digital audio 510, background images200, foreground images 202, text 208, digital media graphics, digitalanimation, and/or branding graphics 206. The user 501 may participate inan online community system 521 in which the server 590 sends the user ID520 to the online community system and receives lists of community userattributes 515 and active vs. inactive status 517. In this way, theserver 590 may determine which users are online and which users may beable to share a personalized digital asset experience. Stored userprofiles 561 are stored and the server 590 may access a user profile 560using a user ID 520.

Content 531 may also be stored and provided to the server 590 in theform of digital graphics or video 530 and/or digital audio 534,optionally based on content requests 536.

A digital asset repository 541 may receive asset requests 540 from theserver 590 and may provide items such as background images 200,foreground images 202, text 208, and branding graphics 206.

In one embodiment, the modified content is stored in modified contentstorage 551 and includes a time index 550, an asset ID 552, a media ID554, a user ID 520, digital video 500, digital audio 510, backgroundimages 200, foreground images 202, text 208, and/or branding graphics206.

In an alternate embodiment, personalized digital media narrative iscreated from the content 531 and the digital asset repository 541. Thenarrative may not be stored as modified content but may be directed atthe user 501 without storage.

FIG. 5B illustrates potential databases that may be used in the creationof the personalized digital media asset and the relationships betweenthose databases. In FIG. 5B, an Uberdirector (Udir) 512 is used tocreate a user profile 561. Security is maintained in the user profile561 through the use of a profile security database 522 and securitymanagement system. The Udir 512 works in conjunction with an I/O map 513to interface to other databases, such as a group and social dynamicsdatabase 518. The group and social dynamics database 518 may permit theuser to interact with other users of the digital media asset todetermine the dynamics between that user and the group. The Udir 512 mayalso work with the digital based evolving nature of the story 516 tocreate the personalized digital media asset. The actual elements used tocomplete the digital media asset may be contained in digital assetrepository 514.

Referring again to FIG. 5B, trigger points 320 may be used to create anexperience 504, which is viewed by an audience 502. Trigger points 320work in conjunction with the digital asset repository 514 to bring listsof online users and their comments 511 in at the trigger points 320.Asset sequencing, timing and security 506 may also play a role indetermining the final digital media asset which is presented as theexperience 504 to the audience 502.

A user profile monitor 500 may also work to understand outside emotionand data mapping 528 to determine whom the user is connecting withonline 526 and the traveling profile management 524, which may ensurethat an individual profile travels from program to program. Each ofthese elements may act to create a more complete stored user profile 561and thus a better customization of the experience 504.

FIG. 6 illustrates exemplary relationships between social, emotional andcognitive elements, all of which may define a user's affinities. As isunderstood by those skilled in the art, a user's reality may bedetermined by a number of elements including: his or her emotionalattraction 604 to certain affinities 601 and elements that arepresented; his or her interaction with others during consumption of anexperience (which is the user's social element 602); and the user'scognitive element 603, which is his or her awareness and perception ofthe world around him or her. These three elements may form the basis fora user's reality as perceived by him or her.

The three elements, as illustrated in FIG. 6, may include elements of an“internal narrative perception identification framework topology,” whichdetermines the user's tendencies, temperaments and provides aclassification so as to increase the digital media narrative impactthrough providing media elements. Other examples of the elements of theinternal narrative perception identification framework (see FIG. 23)include social/collective attributes, time sensitive, episodic andexpectation sequencing, self-narrating content classification, theuser's tendency to be literal vs. metaphorical, an identifier vs. adetacher, or an engager vs. a bystander. These elements of personalityare well known by those skilled in the art and are exemplary only. Otherattributes may be used to define the user's internal narrativeperception identification framework topology.

Referring to FIG. 7, exemplary components of the cognitive 603 elementmay include: analytical skills 702 or the ability of a user to becomeconsciously aware of elements that are being delivered and to analyzeelements that he or she perceives; verbal processing 704 or the abilityto verbalize his or her reality; inferencing 706, in which individualsinfer the meaning of certain elements based on other elements;visualization 708 of elements of the reality; the understanding andreception of speech 710; the user's ability to compute things 712; theuser's ability to communicate in written language and/or acquire newinformation 714; the user's preference in method of acquisition ofinformation 716; the users' ability to reason by analogy 718; and theuser's ability to quantify 770.

FIG. 8 illustrates exemplary social elements 602 and their componentstherein. These components may include: the groups 802 that the user isaffiliated with; a social perception identification framework 804, suchas a user's on-line personality or alter ego; the social personas 808,which are how people perceive that user; a user's social affinities 806;the level of involvement 810 that the user has with other individuals;the relationship 812 the user has with other individuals, the modes ofinteraction 814 or how the user communicates and interacts with otherindividuals; the ability of the user to be apt or, alternatively, ineptin performing functions, such as social interaction 816; the attitudes818 the user has; and internal narrative perception identificationframeworks 820.

FIG. 9 illustrates exemplary components of affinity 601, includingcultural affinities 902 in which the user is associated with aparticular culture or race; artistic affinities 904; his or her digitalmedia narrative (entertainment) likes and dislikes 906; his or hergeographical affinities 908; the dates and events that are important tothat user 910; his or her sensation and perception of the world 912, hisor her iconographic perception of the world 914; and his or herindividual affinities 916. These affinities may determine how a userperceives the world and may represent the particular elements that allowa user to influence other human beings. Users perceive or receive asensation, consciously or unconsciously create the basis for emotions,and provide a catalyst for thoughts and emotions that are stored in thebrain through an n-Gram encoding, ultimately placing the experience in auser's memory. By having knowledge of a user's affinities, it may bepossible to influence the user by creating a closer bond throughpersonalization to make a narrative experience more meaningful to thatuser.

FIG. 10 illustrates a detailed flow chart for an embodiment of thedevelopment of the components for a personalized digital media asset.Referring to FIG. 10, the creative director 110 may watch and listen tocontent in step 1000 and subsequently tag the time indices withaffinities in step 1010. A test 1020 may then be performed to determineif sufficient tagging has taken place. This step may include comparingthe number of tags set to a minimum threshold value or determining ifall of the tags determined by a test sequence have been set. If furthertagging is needed, a return to step 1010 occurs. If sufficient tagging1020 has occurred, the affinities may be edited and associated withcertain digital media assets in step 1030. Step 1030 may represent theaffiliation of tags to trigger points 320 having certain affinities,such that based on a user's profile and interaction it becomes possibleto retrieve the appropriate components of the digital media asset 212 tocreate a personalized experience. Subsequent to the creation of theaffinities, the tags may be stored in step 1040 and the interactions maybe created in step 1050, which may result in the development ofcollateral materials for narrative marketing in step 1060.

FIG. 11 illustrates exemplary tables for a user profile when the userprofile is stored in a relational database. A first table 1101 maycontain a user ID, password, and e-mail address of a user. A secondtable 1111 may contain the user ID, a particular profile element, aknown/unknown status field, and an importance field. The second table1111 may tabulate profile elements pertaining to the user. Profileelements may include any number of affinities previously discussed,internal narrative perception identification framework profile elements,or any other attributes of the user. A third table 1121 may contain theuser ID, one or more profile elements and a value ranking for eachprofile element. The value may indicate that the user has an affinityfor that particular type of profile element and the ranking of thatvalue. As illustrated in FIG. 11, aspects of the user's affinity for artmay be known. These aspects may have been determined by one of themethods detailed earlier. In the example shown, it is known that theuser has an affinity for abstract and renaissance art. In oneembodiment, the rank column of the third table 1121 is used to representthe strength of a positive affinity. In this embodiment, the rankingindicates that the user likes renaissance art more than abstract art. Inanother embodiment, the rank column of the third table 1121 is used toillustrate how strong the affinity is with that particular value. Insuch an embodiment, higher rankings do not indicate stronger positivepreferences, but rather indicate stronger impact data structures forrelational databases used to link components for personalized digitalmedia assets with affinities.

FIG. 12 illustrates an object-oriented approach for storing user profileelements and linking components of a personalized digital media assetwith affinities. Using an object-oriented approach, a person object1200, containing a profile of elements and set element commands, may becreated and related to a people group object 1210. A people group object1210 may also be related to an experience object 1220, which containsaspects of the profile element including the name and description of theprofile element as well as its relationship to other objects, whichallows retrieval of recorded affinities 1230. Individual affinities 1240may be separate objects that contain specific elements known to beimportant to that user.

The database structures illustrated in FIGS. 11 and 12 are exemplaryonly, and illustrate certain aspects of the user profile and his or heraffinities for certain objects and shared experiences that are part ofhis or her social interactions. Regardless of the particular databasestructure used, some or all of the aspects of the user's realitydescribed earlier may be captured in a database to permit a user'sprofile to determine digital media assets that have a strong impact onthat user. The optimization process of finding the strongest or mostappropriate affinities and best match to the user's internal narrativesocial perception identification framework 804 may be based on a numberof algorithms. Exemplary algorithms may include look-up tables, in whichvalues of profile elements are matched to digital media assets, andcorrelation algorithms, which correlate profile elements, values, andranks with profile elements, values, and ranks for a digital media assetto determine the best digital media asset to present. Other techniquesfor matching the user profile to the digital media asset may includeneural networks and fuzzy logic, wherein aspects of the user profile areused to train the network or as inputs to the fuzzy logic system todetermine the best digital media asset. Other types of artificialintelligence techniques, well known to those skilled in the art, mayalso be used to find the digital media asset, or sets of digital mediaassets, that have the largest impact on that particular user.

FIG. 13 illustrates a block diagram of a computer system for arealization of the server 590 based on the reception of multimediasignals from a bi-directional network. A system bus 1320 transports dataamong the CPU 1312, the RAM 1308, Read Only Memory—Basic Input OutputSystem (ROM-BIOS) 1324 and/or other components. The CPU 1312 accesses ahard drive 1300 through a disk controller 1304. Standard input/outputdevices are connected to the system bus 1320 through the I/O controller1316. A keyboard may be attached to the I/O controller 1316 through akeyboard port 1336 and the monitor may be connected through a monitorport 1340. A serial port device may use a serial port 1344 tocommunicate with the I/O controller 1316. Industry Standard Architecture(ISA) expansion slots 1332 and Peripheral Component Interconnect (PCI)expansion slots 1326 allow additional cards to be placed into thecomputer. In an embodiment, a network card is available to interface alocal area, wide area, or other network.

Software to provide the functionality for a personalized digital mediaasset creation may be developed using a number of computer languagessuch as C, C++, Perl, Lisp, Java and other procedural or object orientedlanguages. Different programming languages may be used for differentaspects of the system, such that a first programming language may beused for the content creation process illustrated in FIG. 10 and asecond programming language may be used for the determination of thedigital media assets to present to the user.

In one embodiment, the software may be a web-based applicationcontaining program modules. The program modules may include Javaservlets, Java Server Pages (JSPs), HyperText Markup Language (HTML)pages, Joint Photographic Expert Groups (JPEG) images, Macromedia FlashMX movies, and/or a reusable Macromedia Flash MX component. The softwaremay be executed on a compatible server environment including a webserver, servlet container, Structured Query Language (SQL) database andJava Database Connectivity (JDBC) driver.

The Macromedia Flash MX movies and the reusable Macromedia Flash MXcomponent may include multiple Macromedia Flash MX source files. Aprogrammer may supply a first file that contains code for a Time Framecomponent and/or a reusable Flash MX component that implements the userside of the trigger point 320. An implementation may include visuallyframing the image to be displayed and resizing the image to be displayedto fit the frame, if necessary. For example, a programmer may supply asecond file that includes code having two Time Frame instances and threebuttons per Time Frame, the buttons including a “Warmer” button, a“Colder” button and a “Reset” button. The “Warmer” button may set avariable indicative of an affinity value to a lower value and load animage (or images) from the server that correspond to the new variablevalue. Similarly, the “Colder” button may set the affinity valuevariable to a higher value and load an image (or images) from the serverthat correspond to the new variable value. The “Reset” button may resetthe variable to a mid-range value or the previously stored value for theuser. As an alternative to the second file, a third file may be storedincluding a Time Frame instance, a “Load preferred image” button and twoor more text entry boxes. The user may utilize text entry boxes toenter, for example, an affinity group name and a username. When the userenters valid information into both text entry boxes and clicks on the“Load preferred image” button, the information may be sent to theserver. The server may use a database table to select an image based onthe received information and may return the selected image to the user.

The application software may include multiple database tables such astables of internal narrative perception identification frameworks,current users, user specific social affinities, user specific emotionalaffinities, and/or trigger points. In an embodiment, the applicationsoftware may include a table that specifies an image that bestrepresents the element for a specific affinity element group andaffinity element.

The application software may include one or more HTML pages used toaccess the Macromedia Flash MX source files and to update the storeduser Profile. The application may include one or more Java servlets. Inan embodiment, a first Java servlet is utilized to find the affinityelements having the maximum value for the specific user, among allaffinity elements in a specified group, and return the imagecorresponding to that element having the maximum value. In theembodiment, a second Java servlet is utilized to display the affinityvalues for the user, the affinity type and the affinity element groupand to provide a means for the creative director 110 to update theaffinity values.

The application software may include a plurality of JPEG image filesthat are provided from one or more sources. The sources may include anypublic source of image files, public copyrighted files with anappropriate copyright agreement or private files

FIG. 14 illustrates the construction of an Interacting Musical Intersode(IMI). When used herein, Interactive Musical Intersode (IMI) may referto an embodiment of a personalized digital media narrative program thatis used to create a personalized internal narrative experience for theuser. Referring again to FIG. 14, a window 1400 is presented whichcontains the IMI 1420 and a toolbar 1410 for the construction of the IMI1420. In one embodiment, the creative director 110/distributor 102 viewsthe elements which comprise the digital media asset repository. In anembodiment, these elements include audio 1430, video option/graphicoption track 1 1432, which comprises background option 1, videooption/graph option track 2 1434, which comprises background option 2,overlay text option 1 1436 and overlay text option 2 1438. In oneembodiment, each of these digital media asset options is linked to anaffinity, such that switching can occur between these elements at theappropriate trigger points 320. In one embodiment, the IMI is realizedusing Flash such that overlays are created and switching occurs betweenbackground overlays and the appropriate audio to create the IMI 1420.

FIG. 15 illustrates an exemplary registration screen of an IMI in whichan audience member is presented with a log on window 1520 in order toview the IMI. In an embodiment, having the user log on creates theability to either retrieve information about that user from the databaseor create a new entry in the database about the user.

FIG. 22A is a flow chart that illustrates an embodiment of the presentsystem at work. A user may access a network or website at step 2200. Thenetwork or website may determine whether the user is a new user or arepeat user at step 2205. This check may include reviewing the user'scookies or simply asking the user to enter user identification (e.g.username and password). If the user is a repeat visitor, the user may beasked to input a user name and password at the login step 2210. The userprofile may then be loaded into the system. If the user is a new user, auser profile may be created at step 2220. The created user profile maythen be loaded into the system. In an embodiment, the created profilecontains at least a username and a password. The profile optionallyincludes demographic data such as the user's gender, age, regionallocation and ethnic background. At step 2230, the user may select adigital media presentation. As the digital media experience begins, thedefault digital media presentation may be presented for the user'sviewing and/or listening pleasure. During the viewing of the digitalmedia presentation, at step 2240 the trigger points 320 may be comparedwith the tags 1010 stored in the user's personal profile to determine ifthe default digital media asset video sequence 300 should be changed. Ifno tag 1010 is present for the tested trigger point, the digital mediaasset video sequence 300 may be viewed unchanged until the next triggerpoint 320. If a tag is present, the content of the digital media may bechanged according to the stored user's personal profile 561.

In one example, an advertiser that manufactures various types of petfood, including dog food and cat food, forms an agreement with a recordlabel that distributes music videos on the Internet for free viewing. Anew music video of a popular artist may include a scene, segment orimage having a dog or cat walk across the background to eat from a bowlof food or simply have a dog or cat graphic. Sitting next to the bowl offood is a bag labeled with one of the advertiser's brand name of petfood. Upon entering the website, the user's personal profile may beaccessed. The profile may include information that the individual is adog lover and/or dog owner. During playback of the video, a selection ofthe species of animal may be determined at the trigger point 320 basedon the viewer's profile. For example, a tag 1010 in the profile mayindicate to insert a dog into the video. Insertion of a dog into themedia as opposed to a cat increases both the effectiveness of theadvertising, by allowing the advertiser to highlight dog food to a doglover, and the enjoyment of the video, since a dog lover is more likelyto enjoy a music-based digital media experience featuring a dog. Thus,both the advertiser and the artist may benefit from the enhanced digitalmedia being presented to viewers. Furthermore, the personal profile mayfurther indicate a preferred breed of dog, such as golden retriever orterrier. If such information is specified, the specific breed of dog orcat may be inserted at the trigger point 320. The affinity of the userto the breed of animal may result in the user feeling more personallyconnected to the video.

FIG. 16 illustrates exemplary digital media assets in the form ofspecialized background materials 200 illustrating one part of the world(the continent of South America) and background materials 200 includinga woman's eye and face. In the event that it is determined from the userprofile that the user has an appropriate (or positive) affinity forSouth America, this background may be selected for presentation.Similarly, the woman's eye (or eye color) and face (or hair color) maybe selected if it is determined that they would create a betteremotional experience for the user. By providing this material to thatparticular user an enhanced emotional experience may occur for the user.

FIG. 17 illustrates a screen with personalized content in the form ofdigital media in which an automobile segment produced by the creativedirector 110 is shown on the screen along with a young man and a youngwoman. In one embodiment, these images are produced in conjunction withthe audio, such that the user hears the artist's song and sees thisspecialized content and background material to create an emotionallyenhanced experience. In one embodiment, the digital assets used tocreate the personalized digital media asset are selected based on theuser profile and the ability to optimize the emotional experience forthe user. These assets may include the make and/or color of theautomobile, the ethnic background of the young man and young women, andeven the color of the eye in the background.

FIG. 18 illustrates a screen posing a question relating to an IMI to auser. In this screen, the user is asked a question 1820 and can respondin textual form. The system may use the response to determine the user'sperception of the IMI and consequently the user's preferences within themedia narrative form. The system may further assess the user's potentialaffinity and add it to the user's profile.

FIG. 19 illustrates a screen posing a multiple choice question 1902relating to a media narrative experience. The user may respond to thequestion by selecting an answer. The selected answer may provideinformation to the system regarding the user's desires, preferredaffinities and other internal narrative perceptual identification basedattributes. In FIGS. 18 and 19, digital media assets are presented onthe screen and may be customized according to the updated user's profilebased upon the answers to those or previous questions posed to the user.

FIG. 20 represents an alternative screen for posing questions 2002within a digital media asset based experience to a user in which theuser is asked to respond to a particular question and a particularcharacter 2004, in this case from a TV series. Upon responding, thesystem may store information about the user and update the experience orpresentation based on the current answer.

FIG. 21 illustrates a screen having a “chat type” window 2102 relatingto an experience in which case the user can communicate with other usersof the same experience. One advantage of this embodiment is that theusers can share their comments on that experience with each other ineither an anonymous or non-anonymous format. In another embodiment,users can simultaneously log on to an experience in a customized format.In this embodiment, the users can communicate with each other through achat type window, e-mail, instant messenger, or other communicationmechanism to discuss their emotional experience and put themselves intothe story or the narrative experience. Such communication may allowusers with higher social tendencies to have an enhanced experience andbe more receptive to the sponsor's involvement as a result.

The screens of FIGs.15-21 may be enhanced by operation of a fuzzy logicbased Enhanced Director Agent. FIG. 22B illustrates an embodiment of theoperation of a fuzzy logic based Enhanced Director Agent (EDA) forDigital Media Assets (DMA) “action” in a dynamically delivered digitalmedia narrative platform. Audience interactions and/or inputs arenormalized in step 2250 and the normalized inputs are stored inindividual profile population files 2260. The profile population perindividual may be used to evaluate inputs and to infer mixture ofdigital internal narrative perception identification framework profileelements, including internal narrative perception identificationframework, per individual in step 2270 and sends this information to thedigital asset evaluation step 2280. At step 2280, the creative director110 evaluates the digital asset classification 2275 to infer a mixtureof DMA asset preferences. This mixture may be used to form a priority ofdigital internal narrative perception identification framework profileelements at step 2285. At step 2290, the priority of DMA action isdetermined using the individual digital internal narrative perceptionidentification framework profile. The DMA may include aspects such asaudio, graphic, animation features, video features, the timing of theaudio and/or video and any other transformable aspect of the DMA. Atstep 2295, the DMA action is set to correspond to each individualdigital internal narrative perception identification framework profileaccording to the highest probability for personal preference andenhanced audience identification.

The DMA action may include changing any aspect of the digital medianarrative that enhances the experience without destroying the integrityof the narrative experience. The action may include time sensitivechanges, such as the changing of events, the playback speed, the timingof playback or the sequence of events such as changing the orientationof “scenes.” The action may include changing the audio including thevolume of playback, the score (i.e., the background music), the languagespoken or even the accent of the speaker. The action may includechanging the video aspect such as the gender, race or age of acharacter, the background scenery, the elements of the episode (e.g., amotorcycle, bicycle or horse is ridden), the color of clothing worn, anovercast or sunny sky, or any other visual aspect of the DMA. Theinvention is intended to cover any DMA actions that make the digitalmedia asset video sequence 300 more connected to the viewer and enhancethe experience.

In an embodiment, the DMA actions are logical and do not break the flowof a narrative or an episodic narrative. In other words, in anembodiment, a changed asset does not destroy the plotline of a story anddoes not introduce a character or element that has no logical reason forappearing in the frame. For example, in this embodiment, it would not beappropriate to change the background scenery to a cityscape if thecharacter is shown wearing skis, conversely changing the background to amountain while the main character is carrying shopping bags woulddestroy the flow of the DMA.

Another collaborative aspect may include enabling another user tocontrol the digital media presented to the user. A first user mayexperience a digital media narrative that includes the aforementionedautomatic enhancements and allows for personal selection of events/mediacontent. The first user may enjoy the content so much that he or shewishes to share the experience with select friends, family orcolleagues. The user may save the personalized digital media and enableselected individuals to share the experience by informing the system.The first user may set a security level for further sharing. A lowsecurity level may allow general access to the digital media narrativeand enable secondary viewers to share the personalized digital medianarrative with other viewers. A high security level may limit viewing ofthe digital media content to users having a direct relationship to thefirst viewer. A medium security level may limit access to viewers havingeither a direct link to the first viewer or an indirect connection, suchas a friend-of-friend connection.

FIG. 23 illustrates a narrative perception identification framework thatenables matching of digital media assets to individuals. The frameworkmay be broken down into, for example, seven subsections: cognitiveaffinities, emotional affinities, social affinities, self-narratingself-maintaining content, internal narrative traits preference topology,time sensitive/episodic and expectation sequencing, andcollective/collaborative. Each of these subsections may be assigned avalue for a particular user. In an embodiment, cognitive affinities maybe assigned a value defining whether an affinity is unknown or whetherthe person has a positive or negative affinity for a digital mediaasset. In an embodiment, unknown, low, medium, and high affinities mayalso be assigned to emotional and social affinities. In an embodiment,the self-narrating self-maintaining content may have no content,appropriate content, non-appropriate content, or the meaning of contentthat populates other fields. In an embodiment, time sensitive/episodicand expectation sequencing may determine whether the sequence alignswith cultural or psychological expectations, speed, motion variance andoutcome variables for outcome expectations, and/or intent valencing andintent association. In an embodiment, the collective/collaborativesubsection may determine whether the individual is socially connectedwith other users, the definition and relevance of such connections, andwhether the individual prefers to receive content from others. Thecollective/collaborative subsection may include a determination ofwhether appropriate content or inappropriate content should bedisplayed.

It should be noted that the invention is not limited to viewing on aPersonal Computer (PC) or laptop computer but is intended for use withany digital viewing or listening device. This includes, but is notlimited to, televisions, Personal Digital Assistants (PDAs), wirelesstelephones, MP3 players and any other device utilized to view or listento video and audio signals and that can carry on two way communications.

The many features and advantages of the invention are apparent from thedetailed specification. Thus, the appended claims are intended to coverall such features and advantages of the invention which fall within thetrue spirits and scope of the invention. Further, since numerousmodifications and variations will readily occur to those skilled in theart, it is not desired to limit the invention to the exact constructionand operation illustrated and described. Accordingly, all appropriatemodifications and equivalents may be included within the scope of theinvention.

Although this invention has been illustrated by reference to specificembodiments, it will be apparent to those skilled in the art thatvarious changes and modifications may be made which clearly fall withinthe scope of the invention. The invention is intended to be protectedbroadly within the spirit and scope of the appended claims.

What is claimed is:
 1. A method for personalizing media content usingsynthesized audio, comprising: generating, by a server, a scriptconfigured to output a digital media stream; capturing, by a computingdevice communicatively coupled to the server, a user interaction withthe digital media stream; analyzing, by the server, the user interactionto determine a user affinity for the digital media stream; selecting, bythe server, an audio asset based on the user affinity; and updating, bythe server, the script to insert the audio asset into the digital mediastream.
 2. The method of claim 1, wherein the audio asset is aspeech-based audio asset.
 3. The method of claim 1, wherein the audioasset is a song.
 4. The method of claim 1, wherein the user affinity isselected from a group comprising at least one of a preference, anemotional value, a cognitive value, and a social value.
 5. The method ofclaim 1, wherein the user interaction is selected from a groupcomprising at least one of a viewing habit, a purchase, a selection, andan answer to a question.
 6. The method of claim 1, wherein the selectingutilizes an artificial intelligence method to identify the digital audioasset that matches the user affinity.
 7. The method of claim 1, whereinthe selecting utilizes a neural network to identify the digital audioasset that matches the user affinity.
 8. The method of claim 1, whereinthe user interaction is an input indicating a like or dislike by a user.9. A system for personalizing media content using synthesized audio,comprising: a content distributor configured to generate a script, thescript configured to output a digital media stream; a server coupled tothe content distributor, the server configured to capture a userinteraction with the digital media stream; and a correlation algorithmcommunicatively coupled to the server, the correlation algorithmconfigured to correlate a user affinity with the user interaction,wherein the content distributor is further configured to update thescript to insert an audio asset into the digital media stream, whereinthe audio asset is selected based on the user affinity.
 10. The systemof claim 9, wherein the audio asset is a speech-based audio asset. 11.The system of claim 9, wherein the audio asset is a spoken prompt. 12.The system of claim 9, wherein the user affinity is selected from agroup comprising at least one of a preference, an emotional value, acognitive value, and a social value.
 13. The system of claim 9, whereinthe user interaction is selected from a group comprising at least one ofa viewing habit, a purchase, a selection, and an answer to a question.14. The system of claim 9, wherein the content distributor utilizes anartificial intelligence method to select the digital audio asset thatmatches the user affinity.
 15. The system of claim 9, wherein thecontent distributor utilizes a neural network to select the digitalaudio asset that matches the user affinity.
 16. A method for providinginteractive synthesized audio content, comprising: generating, by aserver, a script to output a digital audio stream in the form of aspoken question; capturing, by a computing device communicativelycoupled to the server, a user response to the spoken question;selecting, by the server, an audio asset based on the user response;updating, by the server, the script to insert the audio asset, therebygenerating a second digital audio stream; and outputting to thecomputing device, by the server, the second digital audio stream. 17.The method of claim 16, wherein the user response is in the form ofaudio captured from a user.
 18. The method of claim 16, wherein the userresponse is the form of an emotion captured from a user.
 19. The methodof claim 16, wherein the user response is an input indicating a like ordislike by a user.
 20. The method of claim 16, wherein the selectingutilizes an artificial intelligence method to identify the audio assetbased on the user response.